In this month’s Biznology newsletter, see what you think about search engines and the way they treat consumers. Check out this opinion piece based on the conference held by Consumer Reports WebWatch just last week.
I was honored to be invited to speak at last Thursday’s conference held by Consumer Reports WebWatch, Trust or Consequence, which brought together several luminaries (and me, too) for commentary on how friendly search engines are to consumers. Jared Spool of User Interface Engineering moderated the panel, which also included Matt Cutts from Google, Danny Sullivan of Search Engine Watch, Dan Thies of SEO Research Labs, Dana Todd of SiteLab, Michael Yang of become.com, and several other search experts.Reports on the conference have concentrated on whether search engines disclose what results they are paid for, but does that mostly miss the point?
Welcome to the June Biznology Newsletter. I come to you each month with a topic at the center of business and technology—this month we’ll look at consumer activist criticism of search engines.
What do we expect from search engines?
Several of the panelists viewed the search engine business as a for-profit industry that by and large exhibits good corporate citizenship. Searchers pay nothing for search and get a lot more than they pay for—this is as close as you get to the proverbial free lunch. Consumer advocates, however, rightly point out that searchers bring certain expectations (perhaps wrongly) to their use of search engines, and that there is a responsibility for transparency. As is often the case, the truth may lie somewhere in the middle, so let’s examine the situation and then see if any solutions emerge.
Transparency
Maybe you’ve heard the term “transparency”—the degree to which a business or a process is open to those who use it. Consumer advocates are big on transparency, and they’re right. You hear the term full disclosure a lot.
And so it is with the search business. In 2002, the U.S. Federal Trade Commission issued a set of non-binding guidelines for what search engines must disclose about their business relationships with their advertisers (which Danny Sullivan provided a detailed report about at the time). And that was good. It is important for searchers to know the paid results from the organic results—they ought to be labelled and they ought to be explained.
In 2004, Consumer Reports followed up with an excellent report that detailed how the major search engines were complying with the FTC guidelines. Some search engines were doing a good job, others less so. And it was good that Consumer Reports did that as well.
So when Consumer Reports invited me to a panel to revisit this topic, with a great moderator and terrific fellow panelists, I quickly agreed. And that was good, too. At the conference, much discussion ensued about a new report, released that day by WebWatch, that updated the 2004 study. It showed that some search engines had improved their disclosure, some had gotten worse, and most had stayed about the same.
Is more disclosure needed?
That was the big question posed to the panelists. If a little disclosure was good, wouldn’t a lot more disclosure be great?
The panelists, especially me, got lots of questions on paid inclusion. Most people, even search marketers, are confused about paid inclusion vs. paid placement. Paid placement is the pay-per-click results called “Sponsor Listings” on most sites. Paid inclusion is money accepted by search engines in return for listing a page in the organic results—it does not help your rankings but it gets you in the index in the first place. Some searchers may also wonder about paid listings, which is how businesses get listed in Yahoo! and other directories that search engines consider to be very important links. It is the rare searcher that fully understands these different ways that money changes hands between search marketer and search engine.
Many folks have suggested that paid inclusion results should be labelled (there’s that full disclosure thing again), but can searchers distinguish between paid placement and paid inclusion? Would they think that paid inclusion results are tainted in the same way that some searchers have less regard for paid placement results?
And, more to the point, are paid inclusion results tainted? Yahoo! is the only major search engine that uses paid inclusion (Google just announced a method of free inclusion), and Yahoo! vehemently claims to treat all results fairly—not boosting the paid inclusion pages. Perhaps the problem is the way Yahoo! charges for paid inclusion. They charge for each click on the result, so that gives them an incentive to boost the ratings to boost their revenue. Again, I don’t think they do that, but their pricing model leads to suspicion.
But the dirty little secret is that it is all infused with money. Should we label every company that pays Yahoo! $300 for a directory link? How about those that hire search marketing consultants? Google, Yahoo!, and every other search engine provides higher rankings based on these payments—not because of the payments themselves, but because of the improved links and content that the payments provide. Disclosure is important, and I applaud Consumer Reports for demanding it, but it does not go far enough. Consumer Reports does not stop at disclosure for health information; they do independent reporting that educates consumers and helps them take care of their health. Web usage is no different.
What must be done?
The basic problem is that consumers expect search engines to provide an unbiased experience akin to a library, not an information marketplace. The fact that search engines get so many people to use them instead of the library speaks to their incredible convenience, free price, and the quality of their results. When people are appalled to learn that money besmirches the results in some way, it is more a testimony to how much more education is required rather than any intrinsic evil on the search engines’ part.
Consumers need education, not just disclosure. Search engines are not publicly funded, and in return for a free service it is not asking too much for searchers to understand their business model, any more than consumers think that a free trade magazine is a public trust just because it is totally supported by ads.
Search engines need to examine models from the publishing industry as a good middle ground. Nearly all newspapers and magazines are advertising-based, with only a small part of their costs covered by subscription prices. Their ads are clearly labelled, but we expect that their articles are unbiased—we expect journalistic integrity.
Journalistic integrity, however we might wish it were different, does not really create unbiased reporting. Even if you believe that professional reporters and editors ensure that their personal views are not reflected in the reporting, it is undeniable that money has an impact on the news. A company that can hire a fancy PR firm can get attention for its point of view, or can create news events with studies, surveys, contests, donations, and many other newsworthy ideas. Some organizations like Consumer Reports run conferences and issue studies to get attention that an organization with less money could not afford to do. And none of that is a horror. Money taints the news we see the same way it taints organic search results.
But disclosure is not the way that we remove the taint in news gathering. We don’t insist that companies divulge what they spent on their PR event or how many speakers they paid to address their conference.
Our news is relatively unbiased because of several factors:
- Journalistic ethics: Reporters and other in the industry have an almost religious fervor about what is allowed and not allowed within their profession. Each news organization has policies covering behavior. Journalism schools also take this subject very seriously.
- Competition: Enough news organizations exist that no single group can control the news. Despite the herd mentality decried in journalism, it is remarkable how much diversity of thought there is in the mainstream media, and the Web is adding more voices every day.
- Economics: While not a perfect arbiter of ethical bounds, news organizations do lose customers if they are continuously sloppy about their reporting. Some people will always prefer the National Enquirer, but by and large the most successful news organizations have high standards.
Can search engines use this same model? I think they can. It won’t work perfectly, any more than journalistic ethics result in a perfect system, but it will be better than relying on disclosure (as important as that is). When I listen to Matt Cutts from Google, I hear the same commitment and pride that journalists have to get the story right, and I am sure that the other major search engines feel the same way. I don’t know if Consumer Reports needs to take the lead to help lay out the ethical standards or whether the search engines could get together to do it themselves, or whether academia will make this part of a profession. On top of these efforts, the competition is definitely there—different search engines provide different organic results for the same query, resulting in a diversity of opinion. And economics has generally rewarded search engines that have clearly labelled their paid results and who have at least attempted to be unbiased.
If you think that disclosure can give you the transparency needed to engender trust in the search engines, I disagree. Search engines are fiendishly complicated and will only get more so. If you think it is hard to figure out why a page gets the #1 ranking today, you ain’t seen nothing yet. Wait until personalized search provides a different #1 result for each searcher, the way Amazon shows different suggestions for each customer based on what they know about you.
Search engines won’t be trusted because searchers understand how they work, any more than Johnson & Johnson Baby Shampoo is trusted because parents understand the ingredients. No, search engines are becoming big brands—brands that become trusted based on everything consumers know about them, from their own experience, to advertising, to journalistic reporting. Consumer Reports could take on search engine ethics in an even stronger way and help educate consumers as to how the search business works—and review search engines as to their effectiveness. If searchers insist on it (and watchdog groups demand it), natural competition and a rise of journalistic integrity among search engine professionals will allow trusted search brands to emerge that are as valuable as CBS News and Newsweek. Yes, I picked those names on purpose. Those companies have made recent missteps, but that is the point. While not always perfect, a public dialogue about what these companies do is what makes the system work.
